Platform Explorer / Nuxeo Platform 2023.11

Bundle org.nuxeo.ecm.core.storage.binarymanager.s3

Documentation

  • README.md

    nuxeo-core-binarymanager-s3

    This addon implements a BinaryManager that stores binaries in a S3 bucket. For efficiency, a local disk cache (with limited size) is also used.

    Be sure to protect your nuxeo.conf (readable only by the nuxeo user) as the file will have your AWS identifiers (unless you are using instance roles).

    Mandatory Parameters

    • nuxeo.core.binarymanager=org.nuxeo.ecm.blob.s3.S3BlobProvider

    • nuxeo.s3storage.bucket : the name of the S3 bucket (unique across all of Amazon, find something original!)

    • nuxeo.s3storage.awsid : your AWS_ACCESS_KEY_ID

    • nuxeo.s3storage.awssecret : your AWS_SECRET_ACCESS_KEY

    If the awsid and/or awssecret are not set, the addon will try to use temporary credentials from the instance role (if any).

    Optional Parameters

    • nuxeo.s3storage.region : the region code your S3 bucket will be placed in. For us-east-1 (the default), don't set this parameter For us-west-1 (Northern California), use us-west-1 For us-west-2 (Oregon), use us-west-2 For eu-west-1 (Ireland), use EU For ap-southeast-1 (Singapore), use ap-southeast-1 For ap-northeast-1 (Tokyo), use ap-northeast-1 For sa-east-1 (Sao Paulo), use sa-east-1

    • nuxeo.s3storage.cachesize : size of the local cache (default is 100MB).

    • nuxeo.s3storage.bucket_prefix : bucket prefix

    • nuxeo.s3storage.pathstyleaccess : if true, configures the client to use path-style access for all requests (default is false)

    Crypto Parameters

    With S3, you have the option to store your data encrypted. Note that the local cache will NOT be encrypted.

    The S3 binary manager can use a keystore containing a keypair, but there are a few caveats to be aware of :

    • The Sun/Oracle JDK doesn't always allow the AES256 cipher which the AWS SDK uses internally. Depending on the US export restrictions for your country, you may be able to modify your JDK to use AES256 by installing the "Java Cryptography Extension Unlimited Strength Jurisdiction Policy Files". See this link to download the files and installation instructions.

    • Don't forget to specify the key algorithm if you create your keypair with the "keytool" command, as this won't work with the default (DSA). The S3 Binary Manager has been tested with a keystore generated with this command :

      keytool -genkeypair -keystore </path/to/keystore/file> -alias <key alias>
          -storepass <keystore password> -keypass <key password>
          -dname <key distinguished name> -keyalg RSA
      

    With all that preceded in mind, here are the crypto options (they are all mandatory once you specify a keystore) :

    • nuxeo.s3storage.crypt.keystore.file : the absolute path to the keystore file
    • nuxeo.s3storage.crypt.keystore.password : the keystore password
    • nuxeo.s3storage.crypt.key.alias = the key alias
    • nuxeo.s3storage.crypt.key.password = the key password

    Server-side Encryption

    S3 allows you to use server-side encryption to encrypt data at rest with keys managed by S3 itself. To activate-this mode, use:

    • nuxeo.s3storage.crypt.serverside = true

    See the related AWS documentation for more.

    If you want to use Server-Side Encryption with AWS KMS–Managed Keys, specify your key id with:

    • nuxeo.s3storage.crypt.kms.key = your-key-id

    See the related AWS documentation for more.

    Enable CloudFront Direct Download

    Please, read carefully the CloudFront documentation to understand how you bind a CloudFront distribution to a S3 bucket. After you created a CloudFront distribution domain bound to your S3 repository. Accessing your objects is not restricted per default.

    You have to enable the restriction viewer access (Look at Serving Private Content throught CloudFront), then each distribution URL must be signed to access the target object.

    You have to set the Query String Forwarding and Caching on all parameters (Look at Configuring CloudFront to Cache Based on Query String Parameters) in order to correctly forward Content Disposition parameters to get a response with a correct filename.

    CloudFront Mandatory Parameters

    S3 parameters (except nuxeo.core.binarymanager) are mandatory, additionally CloudFront requires some new ones:

    • nuxeo.core.binarymanager=org.nuxeo.ecm.core.storage.sql.CloudFrontBinaryManager
    • nuxeo.s3storage.cloudfront.privKey: the absolute path of the private key file (.pem or .der). Read: Creating CloudFront Key Pairs for Your Trusted Signers
    • nuxeo.s3storage.cloudfront.privKeyId: the private key id.
    • nuxeo.s3storage.cloudfront.distribDomain: the distribution domain name.
    • nuxeo.s3storage.cloudfront.protocol: the prefered protocol (default HTTPS)
    • nuxeo.s3storage.cloudfront.fix.encoding: Enable a workaround to fix an error on CloudFront side (default false)

    Building

    mvn clean install
    

    Testing

    Running the unit tests requires some environment variables and System properties to be set, otherwise they are skipped.

    Environment Variables

      AWS_REGION=eu-west-3
      AWS_ACCESS_KEY_ID=******
      AWS_SECRET_ACCESS_KEY=******
      AWS_ROLE_ARN=arn:aws:iam::783725821734:role/nuxeo-s3directupload-role
    

    System Properties

    mvn test \
      -nsu \
      -Dnuxeo.test.s3storage.bucket=nuxeo-platform-unit-tests \
      -Dnuxeo.test.s3storage.transient.bucket=nuxeo-platform-unit-tests-transient \
      -Dnuxeo.test.s3storage.policy.bucket=nuxeo-platform-unit-tests-policy \
      -Dnuxeo.test.s3storage.bucket_prefix=BUCKET_PREFIX \
      -Dnuxeo.test.s3storage.provider.test.bucket_prefix=TEST_BLOB_PROVIDER_PREFIX \
      -Dnuxeo.test.s3storage.provider.other.bucket_prefix=OTHER_BLOB_PROVIDER_PREFIX
    

    Deploying

    Install the Amazon S3 Online Storage Marketplace Package. Or manually copy the built artifacts into $NUXEO_HOME/templates/custom/bundles/ and activate the "custom" template.

    About Nuxeo

    Nuxeo dramatically improves how content-based applications are built, managed and deployed, making customers more agile, innovative and successful. Nuxeo provides a next generation, enterprise ready platform for building traditional and cutting-edge content oriented applications. Combining a powerful application development environment with SaaS-based tools and a modular architecture, the Nuxeo Platform and Products provide clear business value to some of the most recognizable brands including Verizon, Electronic Arts, Sharp, FICO, the U.S. Navy, and Boeing. Nuxeo is headquartered in New York and Paris. More information is available at www.nuxeo.com.

Resolution Order

[127, 830]
The resolution order represents the order in which components have been resolved by the Nuxeo Runtime framework. This range represents the minimal and maximal orders for this bundle's components.
You can influence this order by adding "require" tags in the component declaration, to make sure it is resolved after another component. It will also impact the order in which contributions are registered on their target extension point (see "Registration Order" on contributions).

Components

Packages

Maven Artifact

Filenuxeo-core-binarymanager-s3-2023.11.13.jar
Group Idorg.nuxeo.ecm.core
Artifact Idnuxeo-core-binarymanager-s3
Version2023.11.13

Manifest

Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven 3.9.6
Built-By: root
Build-Jdk: 17.0.11
Bundle-ManifestVersion: 2
Bundle-Version: 2023.11.13-t20240501-060017
Bundle-SymbolicName: org.nuxeo.ecm.core.storage.binarymanager.s3;singlet
 on:=true
Nuxeo-Component: OSGI-INF/s3binarymanager-managment-contrib.xml,OSGI-INF
 /s3web-request-controller-contrib.xml,OSGI-INF/configuration-properties
 .xml,OSGI-INF/bulk-config.xml

Exports

Charts

    Raw Data: Json Contribution Stats

    Contributions by Code Type

    Loading data

    Contributions by Target Extension Point

    Loading data

    Contributions by Studio Source

    Loading data